Requirements engineering in a highly complex, almost chaotic environment — A case study about the Backlog Refinement in Scrum at an airport!

15 min readSep 2, 2022

A case study by Nils Hyoma, Jan Schneider and Wuk Petrovic.

“Backlog refinement takes you from Vision to Value”

[1], this is how Ellen Gottesdiener and Jeff Sutherland describe Backlog Refinement. For Jeff Sutherland, one of the two founders of Scrum, Backlog Refinement is the backbone of high-performance teams (HPT). But what makes these teams special and what advantages do they bring? Focusing strongly on achieving their goals, they deliver significantly more valuable results than less goal-oriented teams. They often exceed the expectations of their organization in the short term [2].

Scrum already has prerequisites to promote the development of HPT: a clear role definition and tangible objectives like sprints and product goals. The retrospective is a powerful tool to remove barriers that stand in the team's way. If the organization and stakeholders support HPT in product development, the development of the team is significantly enhanced. But how does the organization set the right goals? Together with users, customers, stakeholders, and especially the Scrum Team. And for all involved people focus on providing product vision should be created before or in the first sprint. For the creation and updating of this vision in the sprint, Scrum has a dedicated process: Backlog Refinement. This is the least described of all events and processes, is this process with a focus on collaboration changes from an iterative to an agile approach. We show the process that is necessary in a complex world in Figure 1. We describe the motivation for the procedure for this in the “Chapter Collaborative Product Ownership through Backlog Refinement”.

Figure 1: Classification of the methods mentioned in this article for a very complex, almost chaotic environment. For less complex environments, more traditional methods for tasking backlog refinement are available.

Backlog Refinement

The 2017 Scrum Guide describes the process in exactly one paragraph:

“Refinement of the product backlog is considered to be the process of adding details to entries, creating estimates, or determining the order of entries in the product backlog. Refinement is a continuous process in which the Product Owner and the development team work together to detail the product backlog entries. When the product backlog is refined, the entries are reviewed and revised. The Scrum Team determines when and how this refinement work takes place. It should normally not take up more than 10% of the capacity of the development team. However, the product owner can update or have the entries in the product backlog updated at any time.” [3]

For newly assembled teams, the freedom that the product development framework leaves is often overwhelming. What they do not specifically consider in the definition: How the requirements to be refined later are found and created. Outside the Scrum Guide, we can find the following elements that make up a Backlog Refinement [4]:

· Estimating Backlog items

· Prioritization of the Backlog

· Splitting up work packages that are too large

· Find and resolve dependencies

· Remove requests that are no longer relevant

· Creation of new requirements

· Gain insights

The goal of Backlog Refinement is to ensure that we fill the Product Backlog with relevant, detailed, and prioritized items. The Product Backlog thus corresponds to the current understanding of the product and its goals. Unlike a more formal “requirements document,” the Product Backlog is a dynamic, emergent set of information. For example, not all user stories at the beginning of the project need to be broken down to a fine level or provided with detailed estimates. However, it is important that enough requirements are available for the teams at all times.

In the next chapter, we want to show how product ownership was collaboratively adopted by the whole team and what positive effects have arisen.

Collaborative product ownership through Backlog Refinement

This approach was based on a challenging initial situation in which the Product Owner was also a managing director of a company with 1500 employees. Because of his commitments, he had significantly less time than his Scrum Teams would have liked and the role actually provides. His knowledge of the desired processes, and the environment did not make him easily replaceable. His influence as managing director brought product development significantly into the orientation of the organization. The product to be developed was an almost completely automated scheduling software for a large airport. This ensures the smooth running of services and has the aim that aircraft are handled as quickly as possible. The closely networked processes, short time slots, unpredictable events, and connection to various third-party systems are permanent challenges in the complex planning of the scheduling algorithm. Any wrong decision can lead to chain reactions, including in the airspace. In the worst case, hundreds of passengers do not arrive at their destination on the planned day.

The product we embed in an environment with dependent products. This means that the entire value chain spans several interacting but independent products. In an analysis of the application domain from the point of view of domain-driven design, we identified disposition as the core domain of the client. Other domains, such as billing we have classified as generic domains. For these, standard products are used by the customer as far as possible. A decisive, unique selling point for the organization is the development of individual software in its core domain. Therefore, the scheduling software of the ground handling services was selected as the leading product. This structuring makes it possible to minimize dependencies at the source code level between the individual teams. This allows multiple teams to develop software in a common domain without having to work on the same Product Backlog or code base.

Define goals through a Product Vision Board

The first step before filling the Backlog with requirements is to develop a vision in or before the first sprint. In order to establish a basis for cooperation, a kick-off workshop will take place between the Scrum team, users, and stakeholders. During the exchange of ideas, we make the first valuable contacts with positive and influential stakeholders and users, who are also important later for the refinement processes and change management. Through the exchange at eye level, ideas and needs of all participants are collected, a common understanding of the product vision is built up and the team commitment is strengthened.

Methodologically, we recommend an adapted Product Vision Board by Roman Pichler (Fig. 2). In the orientation to this scheme, both target groups and needs are discussed and selected. In the next step, the team then agrees on the product, product, and sprint goals. This gives the team a clear focus through coordinated prioritization.

Figure 2: Product Vision Board in Miro [5]

In the next step, we must then fill the product backlog with requirements that meet the product goals. Requirements that do not meet these requirements can be briefly documented, but are not the focus of the Scrum Team. There are good experiences with technical product goals that leave the solution space open and put the problem in the foreground. Examples include the “abolition of radio traffic in standard bus procedures” or the “automated scheduling of buses”. The former goal leaves it open whether we will achieve this through software development, change management, or a combination.

Based on the highest prioritized sprint goal (here exemplary the goal “Abolition of radio traffic in standard bus processes”), it is now necessary to identify requirements and refine them continuously. Job shadowing is ideal for identification and a better understanding. Experience has shown that means of tools that support cooperation in the team, but also in the entire company can solve well the formulation and needs-based cutting of user stories. In our experience, Jira (Fig. 3) is ideal here, as it is designed for agile working and can be adapted to the individual requirements and working methods of the team.

Job-Shadowing

Accompanying employees at work Job Shadowing is an effective tool to build up knowledge about the respective domain with — usually little effort. Depending on the situation, it is also possible in Job Shadowing to try out or at least observe work steps yourself. An understanding is built up of how processes and workflows are really carried out, which problems can occur and how they have been solved so far. The method is actually intended for personnel development, but can still be used just as well in determining requirements. A positive side effect can be the building of trust between workers and developers (Fig. 4). Imagine you are a loader at an airport and a developer unloads an airplane with you. The employees will talk directly with the developers openly and honestly about the requirements for the software. Based on the exchange of information between developer and worker, a future-oriented strategy is developed. This helps to use the input gained effectively in product development.

From (domain) storytelling to user stories

The knowledge gained in the previous step must be recorded and confirmed. In a complex domain, it is almost impossible to accurately define business processes. In order to enable collaboration between the Scrum Team and the experts, storytelling [6] is a good idea. We always structured chronologically a story in storytelling and the credo applies: the more accurate, the better. This makes them easy to understand even for people without an IT or process management background.

Domain storytelling as a method uses a simple visual language. This allows stories about applications to be told and documented. The method relies on three types of symbols, which are represented by icons and arrows: actors, work objects, and activities. If the team adheres to the procedure, the derivation of user stories is usually very trivial.

Figure 5: Concept of the Minimal Viable Product based on Domain Storytelling

In order to gain a common basic understanding of the software to be developed, the so-called “Happy Path” is a good idea. This is a simple process in which everything works with no problems. It thus forms the basis for understanding the purpose of the lived process. Subsequently, further critical stories are told based on one’s own experiences and documented as a domain story. Through the outlined and really experienced stories, all users can confirm the meaningfulness and correctness of requirements or express concrete change requests.

Figure 6 shows a lived process without a tablet and scheduling software. It forms the basis for a discussion on the desired digitization. Figure 5 shows how domain storytelling has been used collaboratively.

Figure 6: Actual diagram of a process section shown with domain storytelling

Figure 7 shows a target diagram. Together, they worked out how digitization could take place without overwhelming the users. We can derive relatively easily user stories from the result.

Figure 7: Target diagram of a process section shown with domain storytelling

Figure 8: A user story derived from a DST diagram in Jira

Planning a Minimal Viable Product

After clarification of the requirements, the procedure has established itself that we identify the decisive components for the desired application. This is called the Minimum Viable Product (MVP) (Fig. 5). We only equipped this with the core functions of the planned software, as the focus is on a quick introduction and then on small experiments that should confirm the value of the assumptions about the product. Early feedback is used to improve MVP iteration by iteration and maximize user benefit and satisfaction.

But how can a product be meaningfully reduced to an MVP? One way: a reduced quality of the user interface, as far as possible no dependencies on third-party systems, and a selection of minimal, but value-adding functionalities, such as the Happy Path. It is important that when choosing the requirements; we planned a solution that does not overwhelm future users and makes the most important use cases tangible. We achieve this if we orient ourselves on the current processes of the users.

Design Studio

Through the previous steps of job shadowing and storytelling, the team has agreed on an implementation sequence of the user stories and a rough schedule. The goal is to release the product in a timely manner and to start experiments. Our scenario is about expert software for the dispatchers of ground handling services, in which functionality has a higher priority than user experience and intuitive usability.

The user stories with the technical and professional requirements are already available and now the team is faced with the challenge of developing a suitable UX/UI design with the stakeholders.

Design Studio is an interactive ideation workshop that complements Scrum and in which common thoughts and ideas on solutions are exchanged and developed. In this cooperation, an understanding of how the product is designed and behaves develops. This starts with the creation of an initial process and goes through the user experience to wireframes of a user interface and is repeatedly supplemented and further developed by sprinter knowledge. Based on the graphical mockups, users and stakeholders can now understand the future product more easily and bring in targeted change requests, concerns, or questions. Through the feedback gained in this process and the joint cooperation, the MVP becomes tangible for all parties involved. The knowledge about the core domain is further expanded through this exchange and the inclusion of different perspectives.

Figure 11: Design Studio — Figure 10: Design Studio

Resolving External Dependencies

In the last phase of Backlog Refinement, the focus is on identifying dependencies across teams and planning how to deal with them. We could divide this into the following steps:

· Further adaptation of the MVP based on product and sprint goals
The Product Owner of the scheduling software works with stakeholders and developers to create a release plan for the MVP or a new module based on prioritized product goals, or, if possible, sprint goals.

· Coordination of cross-team sprint goals
We presented the sprint goals to the Product Owners from the other domains in a release plan. If the product of the core domain has requirements for other products, we synchronize the sprints.

· Detailed planning with several Scrum Teams
After harmonizing the sprint goals by taking part Product Owners, the developers were involved. These clarity with each other whether the sprint goals are achievable or need to be adjusted despite dependencies.

· Implementation of the requirements in the sprint
The teams implement the requirements with dependencies at the same time and as cross-team pairs. This immensely reduces the need for documentation for further planning. We can thus present finished increments to Product Owners or stakeholders during the sprint. We clarified technical questions in the pair.

· Joint sprint review
A joint sprint review of the taking part teams completes the end of the sprints outlined above, in which the status quo of the development becomes visible across domains. The sprint review for several groups of stakeholders by focusing on interconnected products is also more tangible and less abstract for the guests.

Experimentation and coaching

After the implementation of the features — partly before the sprint review — the changes were incorporated into the production. The minimal but constant changes lead to the fact that the users are constantly involved, but not overwhelmed. For critical adjustments, the Scrum Team went to the control center of the ground handling services and decided with the team there on site.

Figure 11: Coaching in the control center

Besides knowledge of the software and the area of application, this also brings other significant advantages:

· Development of knowledge among the developers about extreme and exceptional situations and about the wishes and needs of the users

· Formation of close familiarity and connection between the Scrum Team and users of the software

· Timely assessment of the criticality of errors and their solutions by the Scrum Team

Despite structured Backlog Refinement, they often found problems in tests — triggered by previously unknown situations. They fix these as a bug in a timely manner or included new requirements in the backlog. If the new feature works stably, we will coach the users.

Added value through collaboration in Backlog Refinement

After we have described in the previous section a sequence of the refinement process in a very complex environment, we will discuss in this chapter the resulting general added value of cooperation in product development.

Sustainable solutions

A conventional solution to a problem is based on assumptions of an ideal world. In complex environments, defined processes but not lived workflows correspond to lived workflows. This leads to problems when a new product is introduced, as the lack of flexibility in the new software means that the processes now have to be lived rigidly. If we include different perspectives in the solution finding — in this case the requirements determination — then this can help to understand the real criteria for a successful product. Through a discussion and an iterative approach to finding solutions, as in Design Studio, innovative, sustainable, and valuable solutions are created. In the long term, sustainable solutions also mean lower risks and costs.

Identification

Innovations are only successful if the company’s employees stand behind it and benefit from it. The surest way to gain this support and consider their needs is to involve them in generating ideas as early as possible. For the product development team, they enable the identification means that the members lead discussions with the stakeholders and users on an equal footing.

Change Management

Because of the early involvement of the users described above and the consideration of their needs in the product to be developed, a transfer of knowledge between all participants takes place casually.

Besides acceptance, introductions in short and continuous cycles are also conducive to the successful establishment of software, so that the users are not overwhelmed. A sprint approach with frequent small releases and feedback loops is ideal for this. Personal contacts, which are established during job shadowing and coaching support this approach, among other things. As a result, everyone involved sees everyone’s needs and takes them into account in the software and beyond.

Unconsciously, positive, agile, and productive change management with the agile team at the core is already carried out in the procedure outlined by us. They often underestimate the early and constant transfer of knowledge to users. Traditional change management methods can strengthen this.

Scrum Events and Artifacts

Through the collaboration of the different groups, we build a common understanding of the core domain among all participants. Based on this understanding, we complemented the Scrum events and artifacts with the common vision. We observed that documentation, such as the deep granularity of user stories or the written planning of how to deal with dependencies, also decreases significantly across different teams without leading to problems. The resulting freedoms in combination with identification and domain knowledge led to greater effectiveness. The Scrum events, such as the sprint planning, could thus be carried out in a much more targeted manner and the time boxes specified by the Scrum Guide were rarely fully exploited. This is also due to the fact that a lot of communication takes place in advance.

Done is Done

By involving influential groups of stakeholders in the refinement process, such as the works council, the Product Backlog items are so well clarified that no further technical and professional control is required after software development. Thus, the Product Backlog Items, if they meet the Definition of Done, can be delivered risk-free.

Conclusion

Figure 12: Pushback vehicle at Hamburg Airport

We have already used effectively the findings from our approach in several teams and projects. In this way, we can see a pattern: the more the developers are involved, the more they can take responsibility and get involved. This increases the reliability, predictability, software quality, and real benefits of the software.

In our case, until the first experiment in the control center, the value of the requirements was only theory, despite collaboration with users and stakeholders. Through validation, deviations and thus also further knowledge about the core domain were discovered. The developers could significantly deepen their domain knowledge.

For the acceptance and thus the successful use of software, the trust of the users in the product is essential. Through the personal exchange between developers and users, fears were allayed and trust strengthened. As with job shadowing, helpful bonds and an understanding of the true needs and thus requirements developed.

In this way, the Scrum Team could create sustainable software in a complex environment that also required constant adjustments in the client’s organization. Through the collaborative refinement process, the team not only developed requirements independently but also started modern change management. This made it possible to make the software more sustainable and to start and accompany a change to the modern and agile culture in ground handling services.

Based on the German Study [7].